Thesis Covariance Regularization in Mixture of Gaussians for High-dimensional Image Classification

OF THESIS COVARIANCE REGULARIZATION IN MIXTURE OF GAUSSIANS FOR HIGH-DIMENSIONAL IMAGE CLASSIFICATION In high dimensions, it is rare to find a data set large enough to compute a non-singular covariance matrix. This problem is exacerbated when performing clustering using a mixture of Gaussians (MoG) because now each cluster’s covariance matrix is computed from only a subset of the data set making the presence of a sufficient amount of data even less likely. The objective of this Thesis is to study the effect of performing a MoG where the covariance matrix is regularized in high-dimensional spaces. Furthermore we present several long-standing algorithms as MoG with various forms of covariance regularization. From this new perspective of MoG with covariance regularization, we begin taking a look at the effect of covariance regularization on MoG classification accuracy. Our secondary objective is to use our results to begin a discussion about the merits of using increased amounts of covariance for MoG on high-dimensional data. We experiment with three different covariance regularization methods What we find is that the simplest form of covariance regularization experimented with in this Thesis results in the best classification of our natural image data sets. Our experiments clearly show that, as the dimensionality of the data decreases, so does the desired or necessary level of covariance regularization. Daniel L Elliott Department of Computer Science Colorado State University Fort Collins, Colorado 80523 Spring 2009